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Guidance method 

Field of the Invention 

The present invention generally relates to the display of electronic 
5 equipment suuh as mobile phones and more particularly to the browsing and 
reading of a large document on a display screen. 



NM/00 5. if \ 3 




Background of the invention 

Today portable elecbonu; equipment, sudi as mobile phunes end 
10 personal digital assistance devices PDA, are getting ever-smaller. However, 
when the size of the portable equipment is reduced also the size of the 
equipment's display is reduced. On the other hand, there is an increasing 
need lu provide seiviues wilh lexl and piulutes lu subscribers. Tlius the 
screen of the display is usually smaller in size than the actual document, i.e. 
15 only a portion of the document is visible at a time. It is clear that reading a 
large document from a small display screen is rather difficult. 

Different kinds of mechanisms have been developed to ease 
reading from a small display screen. Some of them are considered briefly in 
the following. 

20 In some small-sized, hand-held devices which can be used while 

being carried, scrolling the view on the display screen is based on the loca- 
tion (at least one of x, y, or z coordinates) and/or orientation (at least one ot 
azimuth, elevation, roll) of the device. In such a motion-control environment, 
the portion displayed on the display screen of the hand-heid device depends 

25 constantly on the movements of the hand. Due to the fact that the hand is 
inaccurate, some unwanted effects arise, such as jitter or bouncing, which 
makes reading even more difficult. The same kind of unwanted effects may 
arise also when the hand-held device is used In a moving vehicle. 

In order to avoid the above drawbacks, a method has been de- 

30 vised whereby the direction of the scrolling the data can be locked. However, 
one drawback with this method is that It allows browsing of the data in one 
direction only, i.e. it does not take into account the direction of the text lines 
in the data. 

Methods with scroll bars permitting movements of the text either 
35 horizontally or vertically do not have jitter. That is, scroll bars can be used to 
permit the user to change the portion of the view on the display screen hori- 
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zontalty by sliding the horizontal scroll bar and vertically by sliding the verti- 
cal scroll bar. However, a drawback with such methods is that scrolling is 
always either horizontal or vertical, i.e. no other direction is possible. 

At the moment there is no method available that reduces the er- 
5 rors arising from the bouncing movements around the text lines when read- 
ing a document on the display screen of a portable device assisted by a 
motinn-cnntrol method. 

Summary of the invention 

10 An objective of the present invention is to provide a guidance 

method for scrolling the display on a screen of a hand-held device having 
motion-controlled scrolling. The view shown on the display is a portion of 
larger content comprising text lines that are too long to be displayed entirely 
on the small display. 

15 The guidance method should be adapted to follow accurately the 

reading direction of the text lines from the beginning of the text lines to the 
end of the text lines. The reading direction should remain as uniform as pos- 
sible during the entire reading event, he, unwanted effects such as jitter or 
bouncing of the text lines should not arise during the reading event. 

20 t A typical situation may be when a user is browsing a large full size 

image or a document by scrolling the view shown on a mobile phone's dis- 
!■ . play using motion-control scrolling. 

"l m The guidance method discovers when the user stops browsing 

• 5 ; and starts to read text lines. Then the need for guidance is examined. If the 

: 25 result of examination indicates that the user is reading the text lines, the 

.*•: scrolling direction is guided to follow the reading direction of the text lines. 

The guidance is invisible to the user. 

The objective is achieved in the manner described in the inde- 
pendent claims. 

30 The guidance method comprises three main steps: 1 ) the direction 

. . of the text lines shown on the display screen Is periodically examined; 2) the 

movements of the mobile phone are periodically examined and, based on 
the movements f it is concluded whether the user is reading the text lines; 3) 
if the result of examination in step 2 indicates that the user is reading the text 

35 lines, the scrolling direction is reinforced in the direction of the text line so 

00. 

• « 

• • • 
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long as the difference between the current scrolling direction and the text line 
direction Is less than a predetermined threshold value. 



Brief description of the drawings 

5 The Invention Is described more closely with reference to the ac- 

companying drawings, in which 



FIG. 1a-b illustrates a page of the document to be road on a display of a 
small portabje device, 
10 FIG. 2 is a flowchart illustrating definitions of the direction of text lines on 
the display. 

FIG. 3 is a flowchart illustrating the method according to the invention, 
FIG. 4 Illustrates the Inventive Idea with the help of a full-size Image to be 
read on a display of a small portable device, 
15 FIG. 5a-c show an example of the determination of coordinates on the full- 
size image, and 
FIG. 6a-b demonstrate how filtering Is carried uul. 

Detailed description of the Invention 

20 The invention can be applied to a terminal having a small display, 

at least one sensor such as a magnetic pulse sensor, a acceleration trans- 
" ducer or gyroscope which is mounted in the back of the display, and a char- 

: ! acter recognition method or, alternatively, some other method for recognizing 

the direction of the text lines. 
: " : 25 The principle of the invention and the embodiments are described 

[\ using a mobile phone as an example, but the same principle can, of course, 

be applied to any kind of terminal. 

FIG. 1a shows a typical situation where a user is browsing a large 
full-size image 10 on the display of a mobile phone, in this context the full- 
30 size image refers to a whole document or page, of which only a part can be 
. . displayed on the actual screen of the terminal. The actual display of the 

mobile phone is illustrated by a dotted square 16 on the document page for 
visualizing the proportions, of the display to the document Thus the user is 
reading a document whose actual size is larger than the physical size of the 
35 display. The document page has been divided into several columns consist- 
inn, of text sections 11 - 13 and non-text sections 14 - 15, such as pictures. 

■ • 
i • ■ 

O e 
**s> 
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Lines in the text column 13 are in a different direction than in the other two 
text columns 11 and 12. 

FIG. 1b is an enlargement of the rinded area in FIG. 1a. As seen, 
only a portion of the date is seen on the display at a time, i.e. a line of the 
5 document does not fit from beginning to end on the display. Scrolling direc- 
tions are shown by double-ended arrows, Other scrolling directions are also 
possible, e.g. in the diagonal direction. 

It is assumed that motion-controlled scrolling is used. In order to 
browse the document, the data Is displayed proportionally to the orientation 

10 or location of the mobile phone. That is, data scrolling in the vertical direction 
when the mobile phone is tilted vertically, and correspondingly, the data 
scrolls horizontally when the mobile phone is tilted horizontally. 

When a piece of Interesting text 17 appears on the display and the 
user wants to go into the matter in greater detail, i.e. to read through the 

15 whole document or a chapter concerning the matter, the portion of the view 
displayed is moved to the beginning of the document or appropriate chapter 
by tilting the mobile phone in a suitable way. For example, a movable cursor 
is used to indicate a position of interest on the display surface by tilting the 
mobile phone. Thus the cursor moves horizontally when the mobile phone is 

20 tilted towards the vertical axis of the display, and correspondingly, the cursor 
moves vertically when the mobile phone is tilted towards the horizontal axis 
of the display. , 

Thus, the display shown on the screen depends on the motion of 
the hand. However, as stated above, hand movements are inaccurate. With- 

25 out accurate guidance, unwanted jitter arises, making the reading uncom- 
fortable and slow. This drawback is eliminated by the guidance method de- 
scribed in more detail in the following. 

The guidance method comprises three main steps: 1) the direction 
of the text lines shown on the display screen Is periodically examined; 2) the 

30 movements of the mobile phone are periodically examined and, based on 
the movements, it is concluded whether the user is reading the text lines; 3) 
if the result of examination in step 2 indicates that the user is reading the text 
lines, the scrolling direction is reinforced in the direction of the text line so 
long as the difference between the current scrolling direction and the text line 

35 direction is less than a predetermined threshold value. 
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FIG. 2 shows as a flowchart an example of the implementation of 
examination of the direction of text lines. 

It is assumed here that a user scrolls documents on the mobile 
phone display screen in a way that is described in association with FIG. 1a 
5 and 1b. 

At stage 20, the first task is to analyze roughly whether there is 
text in the current portion on display. If no text is detected, free scrolling of 
the data is allowed at stage 21, and the control is returned to stage 20. 

I here are numerous implementation alternatives tor analysis. One 
10 alternative is that the portion of the data that is seen on the display screen is 
first analyzed by defining the major components, i.e. whether the type of 
component represents the text or non-text- Some character recognition 
method such as an optical character recognition method OCK, can be used. 
In some cases information about the type and size of the document, and the 
15 type and size of the font may already be included in the document, in which 
case no character recognition is needed. 

At the next stage 22, the direction of the text lines is defined. In 
FIG. 1a most of the page is covered with text When this page is analyzed by 
the character recognition method the direction of text lines is defined to be 
20 horizontal. However, if the column 13 had boon shown on the display, the 
text lines would have been defined to be vertical. 

One alternative for the direction definition of the text lines can be 
created on the basis of the middle point of the current portion of view dis- 
played. This method will bo considered later in greater detail. 
25 Another alternative utilizes Image analysis and character recogni- 

tion. The direction is defined on the basis of features of the analyzed data. 

The definition results concerning the direction of the text lines are 
saved at stage 23. The information also includes current time information. 
The above-described analysis and definition are came out periodically at 
30 certain intervals. 

A flowchart in FIG. 3 is used to demonstrate how the need* for 
guidance is examined and carried out while one is browsing a large docu- 
ment un Ihe display surcen or a terminal by scrolling the display. 

Coordinates of the current partial view of the full-size image are 
35 read at the fixed reading point (xi. yi) of the display. In this example, the fixed 
reading point is the middle point of the display. 
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The coordinates of the current view at the middle point of the dis- 
play are then saved at certain time instances e,g. every second, at stage 30. 
With the aid of the set of coordinates, a set of motion vectors is obtained. 
The motion distribution as a time function is calculated at stage 31 by using 
5 the motion vector set. 

At stage 32 the need for guidance is examined. 
If tha distribution is narmw, it means that movements hava been 
found, which do not deflect much from the main movement direction or scroll- 
ing direction. In other words, if the scrolling direction is in the direction of the 
10 . x-axis, the coordinates of the page, which have been read at the middle point 
nf the display, seem to fhllnw ttie genera) riirantinn nf the x-axis (the text 
line), i.e. the scrolling direction Is not exactly rectilinear. If such movement as 
descnbed above is not detected or the distribution is relatively broad, free 
scrolling is still allowed, at stage 35. Otherwise, the next stage 33 is to exam- 
15 ine more accurately whether there is a need to guide the reading direction. In 
this case the distribution is compared with the saved information (see FIG. 2 
stage 23) about the direction of the text lines. At stage 34 ft is analyzed 
whether the result of the comparison indicates, in a given time, successive 
movements which are roughly in the same direction- If the result does not 
20 indicatq such movements, free scrolling of the data is still allowed at stage 
35. Otherwise, the data is filtered in such a way that the direction of the text 
/ lines are taken into account at stage 36. Regardless of the result of the com- 

'I parison, the steps above are repeated from stage 30. 

1 FIG. 4 demonstrates how the defining and filtering are carried out 

25 mathematically. The assumption in this example is that the direction of the 
text lines on the display is defined to be horizontal by the character recogni- 
* tion method. A vector 42 represents that direction. The additional assumption 

. is that coordinates of the partial view 40 of the full-size Image are read at the 
middle point 43 of the display. Based on the successive coordinates, a set of 
30 motion vectors are obtained, wherein if the directions of said motion vectors 
are roughly in the direction of the text line, this indicates that the user is read- 
ing the text lines, The curve 41 represents the set of motion vectors. 

* » 

The similarity of directions between the vector 42 and curve 41 is 

examined. 

35 One way of examining the similarity is described in the following. 

Coordinates of the partial view of the full-size image are controlled by a cer- 
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tain time window. In other words while the document is scrolled the position 
of the middle point 43 on the full-size image 10 varies as time function. 

The curve 41 is sampled by a calculating unit using a sample rate 
FS (e.y. FS=1 s). Each sample uunstels of the coordinates (xi, yi) of a point 
5 Pi on the curve at a given time. For example, a current coordinate pair (xi.yi) 
is saved every seoond within a certain time interval, such as 10 s. These 
samples formulate a temporal matrix T, where the first row of the matrix 
represents Die x-uuurdinales arid the second row Ihe y-coorrJinales, so lhal 
each column presents pne coordinate pair (xi.yi). 
10 The parameter D represents the direction (angle) of the text line 

under examination. The definition value for D is 0 radians when the text line 
is in ihe vertical direction and i\J2 radians when the text line is in the horizon- 
tal direction. Of course, parameter D can have any angle value between 0 - 
n2ft radians. 

15 The current direction (angle) between each consecutive point in 

temporal matrix T is calculated. For example, if at time t1 the coordinates at 
point P1 are (xl, yi) and at time t2 at point P2 (x2, y1), the vector between 
the two points P1 and P2 defines the direction (angle) at time U, where At is 
(t2-t1,t2>t1). 

20 In general the result of calculation Is N-1 directions or vectors 

when the number ot samples is N. The calculated N-1 directions formulate a 
vector U. Here within 10 seconds the result is nine vectors, each of which 
has a direction (angle) between 0-n2n radians. That is the result Is the distri- 
bution of directions within 10 seconds. When the shape of distribution of the 

25 directions is narrow, a lot of movements have been found In the same direc- 
tion. Correspondingly, when the shape of distribution is broad, only some 
movements are in the same direction. 

The similarity of curve 41 and vector 42 is examined by comparing 
the above vectors In the following way. 

30 First a comparison is made as to whether the absolute value of 

the difference hetween the mean of vector U and parameter D is smaller 
than the predetermined threshold parameter V and whether tho standard 
deviation of vector U Is smaller than the predetermined parameter X using 
the following equations 

35 Abs(Mean(U)~D)<V (1) 
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Std(U)<X (2) 
If the results are smaller in both comparisons, curve 41 and vector 
42 are in the same direction. Otherwise curve 41 and vector 42 are in differ- 
ent directions. 

5 In the first case mentioned above, I.e. when curve 41 and vector 

42 are in the same direction, the scroll direction is guided to follow accurately 
the reading direction of the text lines by filtering from the scrolling motion 
those components which differ from the direction of the text lines. 

The effect of filtering Is seen In FIG, 4 with the help of curve 44 
10 and vector 45. Curve 44 demonstrates movement of the page before filtering 
and vector 45 represents movement of the page after filtering. Thus, filtering 
provides for movement of the portion of view in the full-size image accurately 
In the direction of the text lines. The result Is that the user Is able to read the 
text lines with ease because unwanted effects are prevented that might 
15 make reading uncomfortable. 

One way to carry out filtering is described with reference to FIG, 
5a -5b and 6a -6b. 

"First, it is assumed that filtering is not in action in FIG. 5a and 5b. 
FIG. 5a illustrates how the view shown on the display looks at time 
20 tl. It is seen that the middle point of the current view 500 is in point P1 at 
Lime 11. Wilhin time (11+ Al) - 11 Ihe display an the screen has been moved 
or the middle point of the previous display has been moved on the full-size 
image from point P1 to point P2. when the current view 500 Is now the view 
501 as shown in FIG, 5b. 
25 FIG. 6a and 6b illustrate a situation similar to what was explained 

above with reference to FIG. 5a-5b, but here the filtering is in action. 

In FIG. 6a the vector a from the initial point P1 to the terminal 
point P2 represents the displacement of the full-size Image within a given 
time. The vector b is a unit vector (i.e. | b | -1) In the direction of the x~axis. 
30 The initial point of vector b is at point P1 . 

In this case it is enough to determine only the component of a 
which is parallel with x-axis, because the text lines are in the direction of the 
x-axis. 

FIG. 6b illustrates said determination. First the projection of a on b 
35 is determined by a scalar product: 
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p = a • b = |a||b|cose (3) 

where p Is the product of the magnitudes of a and b and the co- 
sine angle 0 between tiiem. In tills example Q Is n/4. Then tho vector a, 
5 represents the component vector of a in the x direction, as denoted by 

a, =pb (4) 

The initial ppint of the vector a* is at point P1 and the terminal 

10 point at point PS. The vector a* represents the displacement of the full-size 
image in the x-directiqn within the time At. 

Tho calculation described above is repeated correspondingly for 
displacements at intervals as long as the filtering is activated. The time pe- 
riod is a sample period 1/FS. 

15 The implementation and embodiment of the present Invention has 

been explained above with some examples. However, it is to be understood 
thai the invention is uul leslriuled by the details of the above embodiments 
and that numerous changes and modifications can be made by those skilled 
in the art without departing from the characteristic features of the invention, 

20 The described embodiment is to be considered illustrative but not restrictive. 
Therefoie, the invention should be limited only by the attached claims. Thus 
alternative implementations defined by the claims, as well as equivalent 
implementations, are included In the scope of the invention. For example, 
guidance can be implemented in any scrolling direction, also diagonally. 

25 Parameters such as the sample rate and the number of samples, as well as 
the time Intervals, depend on the application used. In some cases it Is also 
possible that a user adjusts some of the above parameters, wherein the 
guidance method reacts differently to a slow user and a quick user. 
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Claims 

1. A guidance method for scrolling data shown on the display of a 
terminal, where the scrolling direction depends on the tilt angle of the termi- 
nal, 

6 characterized by the steps of 

reading the coordinates of the scrolling display at a fixed reading 
point rrf the screen at predeterminer! time intervals, wherein a set nf mntinn 
vectors are obtained, 

determining the scrolling direction based on the set of motion veo- 

10 tors, 

examining whether the' scrolling direction is roughly the same as 
the direction of a text line, and if so, 

reinforcing the displayed scrolling in the direction of the text line as 
long as the difference between the current scrolling direction and the text line 
15 direction is less than a predetermined threshold value. 

2. The method according to 1, characterized further by 
comprising the steps of: 

detecting periodically whether text Is shown on the display, 
defining the direction of the text lines in response to the detection 
20 and saving the rcsuit together with the detection time, 

calculating the motion distribution as a time function, by using the 
motion vector set, and 

comparing the motion distribution with the saved result. 

3. The method according to 1, characterized in that the di- 
25 rection of text lines is defined by character recognition. 

4. The method according to 1, characterized in that the di- 
rection of text lines is defined on the basis of the coordinates. 

5. A hand-held terminal having a motion-controlled scrolling op- 
eration, 

30 characterized in that the terminal is adapted to: 

read coordinates of the scrolling display at a fixed reading point of 
the screen at predetermined time intervals, wherein a set of motion vectors is 
obtained, 

determine the scrolling direction based on the set of motion veo 

35 tors, 
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examine whether the scrolling direction is roughly the same as the 
direction of a text line, and if so, 

reinforce the displayed scrolling in the direction of the text line ass 
long as tho difference between the current scrolling direction and the text line 
S direction Is less than a predetermined threshold value. 

6. The hand-held terminal as in claim 5, 

characterized in that the user terminal Is further adapted to 

detect periodically whether text is shown on the display, 

define the direction of the text lines In response to the detection 
1 0 and save the result together with the detection time, 

calculate the motion distribution as a time function, by using the 
motion vector set, and 

compare the motion distribution with the saved result 
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(57) Abstract 

This invention relates to the scrolling and reading of a 
larger document on the display of small hand-held elec- 
tronic equipment, such as mobile phones or personal 
digital assistance devices, where the view on the display 
screen is based on a tilt angle of the equipment. An objec- 
tive is to provide a guidance method that enables reading 
text lines in a document whose actual size is larger than 
the screen of the display, without unwanted effects such 
as fitter or bouncing. Thus the guidance method is 
adapted to follow accurately the reading direction of the 
text lines from the beginning to the end of the text lines. 



(Fig. 3) 



VAST.OTTO 02-09-02 12:29 



KENELLEPATREK Asiakaspalvel 




VAST.OTTO 02-08-02 12:28 MISTA- 



KENELIEPATREK Aslakaspalvel SIVU 014 



2. Sep. 20C2 12:43 



PI I ■ 1/00 «• IU/ 10 



2/6 

LH 




*9 9 
* * 



* 



VAST.OTTO 02-09-02 12:29 UISTA- 



KENELLEPATREK Asiakaspalvel SiVU 015 



2. Sep. 20C2 12:43 



• 



n r • i iqo »• i u/ i 



5/« 



Filter the data 20 that the noveneitt 
being examined is rainfbpe j In the 
dlreotfoflcfrtietext 



Save peridCwfly the 
ecordmstescftta 
view of tfcefidWze image 



r 



30 



as ttoe function 



31 




Compare If the distribution and 
saved Information Indicate 
movements general In the same 
direction 




Enable free 
strolling of data 



FIG.3 



VAST. OTTO 02-08-02 12:29 



MISTA- 



KENELLEPATREK As iakaspal vel SI VU 01 6 



2. Sep. 20C2 12:43 



* 



nr.f/oo s- i // 1 o 



4/4 



LI 




VAST.OTTO 02-08-02 12:29 MISTA- 



KENELIEPATREK Asiakaspalvel SIVU 017 



2. Sep. 20C2 12:43 



$/6 

1/ 



It 



This k flie eittffiiH dli|ilay on tb scram 



PI 

Tlie « tbs nnwirt diepUy on flu terun 



500 



FIG. 5a 



tt+A* 

j'e'flia'eurraftl UteplaY on fljo" i 




FIG. 5b 



VAST.OTTO 02-08-02 12:29 



MISTS- 



KENELLEPATREK Aslakaspalvel SIVU 



2. Sep. 20C2 12:44 



# 



PI f. 1/ 00 S« 10/10 




ti CO 



VAST.OTTO 02-09-02 12:29 MISTA- 



KENELLEPATREK Asiakaspalvel S1VU 019 



